REMARKS 

The Office Action has been carefully considered and the Examiner's comments 
duly noted. The Reconsideration of this application in the light of the amendments for the 
specification, amendments for the claims is respectfully solicited. 

A separate request for three-months term extension has been submitted. If for some 
reason this become the attached, please use this as your authorization to charge the account 
50-1529. 

It is noted that Claims 1 to 14, 19 to 22 and 24 to 28 would be allowable. The 
claims now in this application have been amended to overcome the Examiner's formal 
objections. 

There are terms which the Examiner has objected to in the Office Action, and for 
this purpose applicants respectfully submit the enclosed papers which indicate the terms as 
being International standardization in the MPEG group. These sheets are headed 
International Organization For Standardization, Organisation Internationale DE 
Normalisation, and more specifically referred to as ISO/IEC JTC 1/SC 29/WG 1 1 N2802 
with the date of Vancouver, July 1999. 

If there are any points outstanding the Examiner is respectfully asked to call 
applicant's attorney in order to do what is necessary to place the application into condition 
for allowance. 
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In view of the foregoing, early and favorable reconsideration in this application is 
respectfully solicited. 



Respectfully submitted, 




J. Harold Nissen 
Reg. No. 17,283 

Enclosures: 

Appendix - claims with brackets and underlining 
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Table V2 - 39 - Probability for arithmetic decoding of spatial scalable binary shape 
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1.1.1.1.1 Border formation 

When decoding a BAB, pixels from neighbouring BABs shall be used to make up the context. For both the INTRA and INTER 
cases, a 2 pixel wide border about t he current BAB is used where pixels values are known, as depicted in Figure 7-1 1 . 
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Figure 7-11 Bordered BAB. A: TOP_LEFT_BORDER. B: TOP_BORDER. C: TOP_RlGHT_BORDER. D: 
LEFT_BORDER. E: BOTTOM LEFT BORDER. U: pixels which are unknown when decoding the current 

BAB 

If the value of conv_ratio is not equal to 1, a sub-sampling procedure is further applied to the BAE borders for both the current 

BAB and the motion compensated BAB. 

The border of the current BAB is partitioned into 5 regions: 

• TOPJ-EFT_BORDER. which contains pixels from the BAB located to the upper-left of the current BAB and 
which consists of 2 lines of 2 pixels 

• TOP_BORDER, which contains pixels from the BAB located above the current BAB and which consists of 2 
lines of 1 6 pixels 

• TOP_RIGHT_BORDER, which contains pixels from the BAB located to the upper-right of the current BAB and 
which consists of 2 lines of 2 pixels 

• LEFT^BORDER, which contains pixels from the BAB located to the left of the current BAB and which consists of 
2 columns of 1 6 pixels 

• BOTTOM_LEFT_BORDER, which contains pixels from the BAB located to the bottom-left of the current BAB 
and which consists of 2 lines of 2 pixels 
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The TOP_LEFT_B0RDER and^fc_RIGHT_BORDEK. are not sub-sampled, and )lBs they are. The T0PJ30RDER and 
LEFT_BORJDER are sub-samp leSBfch as to obtain 2 lines of 16/conv_ratio pixels anSRolumns of 16/convjratio pixels, 
respectively. 

The sub-sampling procedure is performed on a line-basis for TOP_BORDER, and a column-basis for LEFTJBORDER. For 
each line (respectively column), the following algorithm is applied: the line (respectively cohuno) is split into groups of 
convjratio pixels. For each group of pixels, one pixel is associated in the sub-sampled border. The value of the pixel in the 
sub-sampled border is OPAQUE if half or more pixels are OPAQUE in the corresponding group. Otherwise the pixel is 
TRANSPARENT. 

The 2x2 BOTTOM_LEFT_BORDER is filled by replicating downwards the 2 bottom border samples of the 
LEFT_BORDER after the down-sampling (if any). 
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Figure 7-12 - Bordered motion compensated BAB. A: TOP_BORDER. B: LEFT_BORDER. C: 

RIGHT_BORDER. D: BOTTOM_BORDER 

In the case of a motion compensated BAB, the bordei" is also partitioned into 4, as shown figure 7-12; 
TOP_BORDER t which consists of a line of 1 6 pixels 

• LEFT_BORDER, which consists of a column of 16 pixels 

• RlGH^BORDER, which consists of a column of 16 pixels 

• - BOTTOM_BORDER, which consists of a line of 16 pixels 
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1.1-1 Spatial scalable binary shape decoding 

1.1.1.1 Decoding of base layer 

The decoding process of the base layer is the same as non-scalable binary shape decoding process. 

1.1.1.2 Decoding of enhancement layer 

When spatial scalability is enabled (scalability is set to 1 and hierarchy type is set to 0) with enhancement type == 0 or When 
spatial scalability is enabled wiih enhancement type == 1 and use ref shape —— Q, scajable_shapej:o_djncLP_rocess is tused fo r 
decoding of binary shape. 

If spatial scalability is enabled, use refshape is sejjo_l_and enhancement type is set to 1. the same non-scalable decoding 
process is applied for binary shape of enhancement layer, In this case, the followjne rules are applied for enhancement layer, 

1 . In PVQP t Inter_shape_coding should be done as bab type of co-located MB in the reference VOP flower lavcr) is 
"OpaQuc". 

2. In BVQP, forward reference VOP. most recently decoded non-empty VOP in the same layer, is always selected as 
reference VQP of shape coding. 

If spatial scalability is enabled, use ref shape is set to 1 and enhancement type is set to 0. then the up-sampled binary shape 
from base layer is used for the binary shape of enhancement layer. The up sampling and down sampling process of this 
purpose also follows up-down sampling method described in the subclause i. 1 . 1 .4. 

When spatial scalability is enabled and enhancement type is set to Q v in rhe enhancement layer, the, forward prediction in P - 
VQP and the backward prediction in B-VQP are used as the spatial prediction. In that oasc die shape information is coded by 
s can interleaving (SI) based method. For the forward prediction in B-VQP a motion compensated temporal prediction is made 
from the reference VOP in the enhancement layer. In that case the shape information is coded hy the CAR method_as_like_m 
base layer except that the shape motion vectors are obtained from those of the collocated bub in the lower layer. Motion vector 
and shape coding modefbab type) of collocated bab in the lower layer are used for decoding the enhancement layer bab, 
The location of collocated bab in the lower layer_canJbe^found by following method, 
collocated MBX 

= min ( max ( 0, current MBX*shape hor sampling factor m/shape ho r sampling factor nV 

NumMBXLower-1 ): 

collocated MBY 

= min ( maxf 0, current MBY* shape yer sampling factor m/shap ever sampling facror n ) t 

NumMB YLower- ! ); 

For the current MB location fcurrcnt MBX._current MBY1. the location of collocated bab in the base layer is denoted *is 
[collocated MBX, collocated MB Y1. current MBX. current MBY. collocated MBX and collocated MBY arc the MD-tmit 
coordinations. NurnMBXLower and NumMBYLower denote the number of micro-blocks in rhe lower layer VQP on 
horizontal and vertical directions, respectively. 

1.1.1.3 Prediction in the enhancement layer 

A motion compensated temporal prediction is made from there fere nee VQP in the enhancement layer. In addition, a social 
prediction is formed from the lower reference layer decoded VOP. These predictions are s c le c ted jn di vidual I y_or_comh inal lo 
forta the actual prediction. 





1.1.1.4 Spatial prediction 

The spatial prediction is mack by resampling the lower reference layer reconstructed VOP to the same sampling grid as the 
enhancement layer. For the resampling, repetition is used on the the lower layer, 

For enhancement layer encoding/decoding, the base layer VP? should be up-sampled as the sampling ratio, which is. included 
in the VOL syntax. In VOL svntax^for enhancement layer, there are four fields for the up-sampling ratio, i.e.. 
shape hor sampling factor n, shape hor samplmg factor m. shape vert sampling factor n and 
shape vert sampling factor n, 
Down sampling 

For the sampling rate, below sampling process is adopted for both horizontal and vertical directions. 
For the down-sampling rate m/ru 

m/n - f l/2 k 1ffm2 k Vn1 » where k is the largest integer that is equal or smaller ihan log ^n/m). 



Input plane 



Linear-down sampling 



1/2 1 



(rr^Vn 



Down-sampled 
plane 



Figure V2 « 1 Down sampling process 

1, Find k = floorf los^n/nti ) 

2, _$ampling^a_pixel_p_e^2^pixel5;_: 

iff x%2 k = (2 k - m Sampling F(x) pixel ; 

3 , On the result of 2. sampling a pixel per n/f m2 k ) pixels. 
Up sampling 

For the sampling rate, below sampling process is adopted for boih horizontal and vertical directions. 



For .the up^saxnpl ingjrate_n/m t 
p/m=fa/<in2*VI f2*\ 



where k is the largest integexj1utjs_e,qua.l or smaller than log ^n/m) 



Input plane 



Linear-up sampling 



n/(m2 k ) 




Figure V2 - 2 Up sampling process 



1 , Find k = floorf loeyfn/m) ) 

2: Repeat_one pixel per (rr^Vfa-rr^^ as follows : 

3. On die result of 2, Repeat 2 k -l pixels per pixel. 

For a case that the sampling ratio is 1/2 for both horizontal and vertical direction, one pixel value in the lower .layer js 
up sampled to 2x2 pixels in the current layer bv the folio wine methods: 

The binary alpha image f n ^ M W h\1xh\ in the enhancement layer is upsampled bv repetition of the pixe 
image f \^ f WA\x,] in the lower layer according to the following formula: 



f /. fO. 1 ) 



In Figure 7-a. the spatial prediction of oneBAB is shown for example. In the enhancement layer the pixel value £ 
[2yj+l]f2xf+l is always same with the corresponding pixel value f\w eT [vt][X(] 
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Collocated block in the reference layer 
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Upsampled BAB for spatial prediction 
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Figure V2 - I o n^K il pin I locations and spatial predict ion ^^ir rent BAB bv repetition 

In the case of arbitrary shape VQ^Bts size and locations change from time to time, In^raer to ensure the formation .of the 

spatial prediction, it is necessary to identify the location of the resampled VOPs in the absolute coordinate system. 

VQP horizontal mc spatial ref and VQP vertical mc spatial ref in the resamplcd reference layer should be coincident with 

those ja_the.ephancment_layer._Her)ce_after the position of the spatial reference of the reference layer is scaled by 

* sampling factor n/+ sampline factor m the resampled reference VQP should be relocated bv referencing the offset values 

that are the differences between the positions of the spatial references of txvo VOPs in the horizontal and the vertical direction, 

Since the shape morion vectors and the shape modes in thereference layer are also needed for tl\e_actual shape coding in the 

enhancement layer bab , the bab type matrix and the morion vector matrix are resampled by repetition. In case of the motion 

vector matrix each components are scaled by the sampling ratio on both directions. 

1.1.1.5 Temporal prediction 

A motion compensated temporal prediction is made from the reference VQP in the enhancement layer. For pred icting a bab, 
the motion vectors of the collocated bab in the lower layer isjosed. Enhancement layer shape motion vector fmvX, mvYI can be 
used by scaling the lower layer shape motion vector fmvXLower, mvYLowerl. The scaling method is as follows: 

mvX = fmvXLower * shape nor sampling factor n)// shape hor sampling factor m 

mvY = (mvYLower * shape ver sampling factor n) // shape yer samplino factor m 

1.1.1.6 Types for enhancement layer bab 

Each bab belongs to one of four types given in Table V2 - 1 , This enh bab type field influences decoding of further shape 
information of enhancement layers. For P-VOP coding only 0 and 1 of the enhbabtvpe are allowed. For B-VQP coding 
scheme, all of the four modes are allowed. 

Table V2 - 1 -- List of enh bab types and usage 



enh bab tvne 


Semantic 


Used in 


0 


intra NOT CODED 


P-/B- VOPs 


1 


intra CODED 


P-/B- VOPs 


2 


inter NOT CODED 


B- VOPs 


3 


inter CODED 


B- VOPs 



When the enh bab type = L the bab is predicted bv upsampline from the collocared block in the lower reference layer and is 
decoded by intra-mode CAE based on Scan Interieavingf SI), 

The enh bab tvpc = 0 means that the error of the predicted bab from. the lower reference layer is less than the threshold. The 
predicted bab is aiven by upsampline the collocated block in the lower reference layer. The spatial prediction method in 
7.5,4.4 or linear repetition method can be used for the upsamplin^The predicted bab is used as the current bab. 

For enh bab tvpe=2 and enh bab tvpe--3. no motion vector is decoded_for the temporal prediction of enhancement layer 

bab. The morion vector of the collocated block in the lower reference layer can be utilized. Each component of the vector 

are -scaled by two for referencing. If the collocated bab in the lower reference )aver_is intra coding mode, the value of the 

motion vector is set to zero. If enh bab rype is decode as "inter NOT CODED", then the error of the predicted bab from 

, the previous VQP is less than the threshold. And the predicted bab is used as the current bab. If enh bab type is decode as 

11 inter CODED", the current bab is decoded by inter-CAE which is used in non-scalable shape decoding. 
1.1.1-7 Decoding bab types for enhancement layer 

In order to decode the bab types, the collocated bab of the lower layer is also examined. Based on the lower layer bab ivpes. 
the following VLC table is used for decoding the enh bab type field. For B-VQP decoding in enhancement layer 
enh bab rype is decoded using in Table V2 - 3. For P-VOP decodine in enhancement laver, there arc only two enh bab type 
fields, i.e., 0 and L and they are decoded by 1-2 bit described in Table V2 - 2. 

Table V2 -2- Decoding of enh bab type for P-VOP in enhancement layer 



enh bab type 


Code 


0 


I 


1 


01 



Table V2 -3 Decoding of enh bab type for B-VQP in enhancement layer 







enh bab type in the enhancement lavcr 




0 


I | 2 | 3 



t • www 



enh bab type 
reference 1 




lower 



or 



(bab type in the base 
layer") 



0(2,3) 


1 


0\ 


■ 


000 


1(4) 


110 


0 


10 


111 


2flU) 


001 


PJ. 


1 


000 


3(5,6) 


no 


0 


111 


1Q 



Note that the numbers 0-3 in these tables correspond to the enh bab types given in 7.8.2.1,6. When the lower layer is_the_base 
laver. rMVDs==0&&No Update". "NfVDs!=0 && No Update" h ("all 0". "all 255"). rintraCAE"! and ("MVDs«=0&& 
interCAE\ "MVPs?=0 interCAE") modes are converted to the enh bab type 2. 0, 1 and 3, respectively. 

1.1.1.8 Intra coded enhancement layer decoding 

Intra coded enhancement layer decoding uses scan interleaving algorithTO.be/ore performing intra-mode CAE. The decoding 
order with SI scanning is as follows : 

1. Copy B from Base layer 

2. Decoding order with Vertical scanning : Vr --> Vpl — > Vp2 —> ... ■-> Vpk 

3. Decoding order with Horizontal scanning : Hr -^>Hp) _--> Hn2 ... --> Hnl 
where. 

B_:_PfexHhatj:anJp^ in the base layer. 

Vertical scanning Pixel of Residual term of vertical lincar-up sampling 
V ^: Vertical scanning Pixel of 2 k term of vertical linear-up sampling 
H f : Horizontal scanning Pixel of Residual term of Horizontal linear-up sampling 
H jj: Horizontal scanning Pixel of 2 l term of Horizontal linear-up sampling 
k : The largest integer that is equal or smaller than 

log^fshape hor sampling factor n/ shape hor sampling factor m) 
1 ; The largest integer that is equal or smaller than 

lp.E ^fs.hape yer sampling factor n/shape yer sampling facror m) 

For an example, when shape hor sampling factor n = 8. shape hor sampling factor m = 3. shape ver sampling factor n - 



5, .shape hpr^sampHng factor rr\=_h the scan order_of p ixels f otdecoding is sliownin Figure V2 - 4 




Figure V2 - 4 ~ An example of decoding order when enh bab type=1 (Decoding order: B V , V ft1 -> H , 




1.1.1.9 Border formation foi^R-a-mode CAE for enhancement layerde 




The border formation for decodingofintra CODED bab i cnh bab tvpe~^l ) is the sameTasihai of the non-scalable sham? 
coding (see the attached file,BoidcT-fonration.doc) exc ept E and U regions in $r2i ^j-^r-§: Sit^r^ t^K. H ie 

border of E and V region of the current bordered bab contaiasjhe pixels from the collocated regions in the no-samnleJ lower 
layer pkne_and_which consists, of 2 lines in bovtom and right of die cuiie.or. bah. 
1.1.1.10 Scan Interleaving Algorithm 

The spatial for resolution) scalability canj?_e_ea5jftyjgpught byincorporatingjhe so-called Scan Interleaving (SI) method. In SI, 
coded-scan-lines are decoded by using the prediction from the closest upper and lower neighboring two linesireference-scan- 
lines)_as_sho^_inJFiigure V2 -_5. Jn the figure, horizontal direction scamiing is described. The pixels on codcd-scan-lincs are 
the pixels to be decoded and the pixels on reference-scan-lines are the pixels that are already reconstructed The_closest upper 
and lower neighboring pixel values in rcfcrence-scan-lmes arc used for reconstructing the current pixeJLfThe closest upper and 
lowe^_nej_ghboTingpjxels_rriean die closest left and right neighboring pixels in vertical direction scanning case.) By using those 
two reference pixels, the current decoded pixel has_tbree^rypes_as follows: 

Transitional sample data (TSD): The closest upper and lower neighboring pixels that are already reconstructed have 
different colors each other. 

Exceptional sample data (ESP); The closest upper and lower neighboring pixels that are already reconstructed are same 
and The current pixel is different with the neighboring pixels. 
Predictable sample data (PSD): The closestupper and lower neighboring pixels thai are already reconstructed arc the same as 
the current pixel. 
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Figure V2 - 5 — Scan Interleaving for Scalable Coding: an example of a times by 2 case. 

To_encoding/decodmg shape mask plane, SI performs the scanning method in vertical and horizontal directions. 

successively. ST with vertical scanning (V-ST) and SI with horizontal scanning (H-SI)_are separately applied to the 
input image, 

For decoding a bab in the current layer, the spatially predicted bab from collocated block, in the lower layer is enhanced bv 
V-Sl. And then H-SI is successively_performed on the enhanced plane of the V-SI. The reconsnycting_order is 
presented in subclause 1.1.1.8. 

In the first step of decoding a bab, a binaryjlag, which denotes the current bab type of SL is decoded bv BaC There are 
two bab types of SI : transitional bab (TSD-B) and exceptional bab (ESD-B). If a bab has no ESP, the bab is sit to 
TSD-B. Otherwise the bab is set to ESD-B. When the flag is '0', the current bab is TSD-B, Otherwise, _the current bab 
is ESD-B. 

In the encoder, for encoding TSD-B. only TSD pixel are encoded bv CAE with the context shown in iL-n-! 3fr£ $) 



%t-§r *r ffi^M t^t.. And for encoding ESD-B, all the pixels in the coded*scan-Iines are encoded bv CAE with rhe 
context shown in .SL-n-! ^Si |£-gr t Sl^M ^K, The encoding order is the same as shown In subclause 



1,1.1.8. 

If a bab is TSD-B, only TSD are decoded in the order of V-Sl and H-Sl. The specific decoding order is presented in 1.1.1.8. 

The decoding process of TSD-B is as follows. 
Following die scanning order, if the current pixel is TSD. then the pixel is decoded by BAC with the conrext in -2.fr>: 



t ift^f 1 -? And if the two neighbor pixels (the closest uppcr(left) and bottomf right)) are the same 



value, then the current pixel has the same value as its neighbor pixels. 
If a bab is ESD-B, all the pixels in coded scan line arc decoded by_BAC with the context in ^^-S: t 

1.1.1.11 Determination of the scanning type in enhancement layer decoding. 

When enh bab type is 1 (intra CODED) and the scale ratio between the layers is square(i,e 200x300 <--> 400x600), before 
performing BAC, the scanning type should be determined. For the determination, the top and left borders of the collocated 
block of the lower layer arc extended with its neighboring pixels like Figure V2 - 6. If an extended border is outside VOP. the 
pixel values of the extended border are set to 2ero. Two counts, N^and Nnare calculated (N. : the number of pixels which have 
different values from the'left neighboring pixel value: N^: the number of pixels which have different values from the up per- 
neighboring value). If Nh is greater than Nt_. the spatially predicted bab for decoding the enhancement layer bab has to be 
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1.1.1.12 



Figure V2 - € - Extension of border pixels. 
Decoding of bab type of SI 



The first decoded symbol of enh binary arithmetic codeO. which performs B AC denotes the current bab type of SI. The 
probability of SI bab type prpbfljriyen in Annex B is used for decoding bab type of SI. When this flag is *0\ the current bab 
is TSD-B- Otherwise, the current bab is ESD-B. 

1.1.1.13 Decoding of intra-mode CAE for enhancement layer 

for decoding intra mode CAE for enhancement layer. BAC using the V-SI and H-SI decoding method is performed. By using 
neighboring 7 bits context a pixel is arithmetic decoded as in CAE, such that: 

Compute 3 context usint* the templates shown in 3fe -r jfewH ^..((a) : for V- Sl, (b): for H -SI ) 

l. 'sinu the context find probability from vhe probability table, ue. ciuy intra ^ urobP for V-Sl and enji _int ra_h _,j>vobf 1 :»>r H-S 
These tables arc given in Annex.B.fsee the attached filg % PTObabj lity-uiblyj 
Decoding the pixel value using the probability. 

The following; subclause describes the computation of the contexts for intra mode CAE. 

1.1.1.14 Contexts for intra-mode CAE for enhancement layer 



Depending on the scanning direction, a 7 bit context C = • 2* is built for each coded pixel as shown in Figure V2 - 7. 

k 

where c^=0 for transparent pixels ando=l for opaque pixels, (a) of Figure V2 - 7 is for the intra-mode CAE with V-SI (Si by 
the vertical scanning), and (b) of Figure V2 - 7 is for tbe intra-mode CAR with H-SI f SI by the horizontal scanning). In f a) of 
the figure, C6, C5 and C4 are the closest decoded pixeLs in top-left top, and top-right location, respectively. C3. and C2 are the 
closest decoded pixels in left and right location, respectively. And CI and CO are the closest decoded pixels in honom -le/t.a n d 
bottom-right location, respectively. 

I n (b) of the figure, C6. C5 and C4 are the closest decoded pixels in top-left, top, and top-right location, respectively. O is the 
closest decoded pixel in left location. And C2. CI _and_C0_are the_clo$esjLd_e_cod.ed _p)xels_in_b_ottom-left, bottom, and bottom- 
right location, respectively. . 
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Figure V2 - 7 - The context information at different scanning of SI 

There are some special cases to consider: 

When building contexts, any pixels outside the bounding box of the current VQP to the left and above arc assumed to be 
zerojtranspatent). 

When building contexts and error resilient disable—O, any pixels outside the space of the current video packet to the left 
and above are assumed to_be_7ero (transparent). 

When, co nstru ct in^ the context, any pixel outs ide the jbo rij crcd B A I > t s a < s u n i M o _bc lj ic. sunur pixel value < » I i I \c v J » 1 v ^ ( 
pnee 1 r uu\\c horde to f the curr ent SAB from the coo text location. 





1.1,1,15 Decoding of^gr-mode CAE for enhancement layer 

For decoding_o_f enhancement laveTwith B-VOP coding, the inter-mode CAE is performed in the case of the "inter coded" 
mode (enh bab type=3). The context infoTmation in Figure V2 - 8 is utilized for this decoding. The decoding process of a 
pixel is done in the same way as non-scalable binary shape coding. 

When constructing the Inter-mode CAE context the following conditional assignment is performed, 
iffel is unknown) c\—c2 
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Figure V2 - 8 - The templates of Inter-mode CAE for enhancement layer. C6 is aligned with the pixel to be 
decoded. The pixel to be decoded is marked with '?'. 



1.1.1.16 



Quality fSNR) Scalable Coding 



Q_uaHtv_SpalabiliTy can be easily sought by spatial scalability. Using the spatial scalability and spatial prediction, binary, shape 
can achieve the Quality. (SNR) scalability. In the decoding process, each spatial scalable layer can be spatially predicted to the 
full resolution. For the binary shape, this scalable structure is equivalent to quality scalable structure. 

1.1.2 Multiple Auxiliary Component Decoding 

Auxiliary components are defined for the VQP on a pixel-by-pixel basis, and contain data related to the video object, like 
disparity, depth, additional texture. Up to 3 auxiliary components (includingLthe grayscale shape) are possible, The number and 
type of these components is indicated by the video object layer shape extension given in *ri-^-£: t 1 



St ItM For examples value '0000* indicates the grayscale (alpha) shape. The same support region as descrj' b_e_d.iii_7.,S.4 is 



used for all auxiliary_components, and the decoding procedure is the same as descr)be_djn_s_ubclaus_esj A A . 1 - 1 . 1 . 1 .6 



